LLM Semantic Conventions: Initial PR #825
Conversation
Adding OpenAI metrics and fixing markdown errors
nirga
force-pushed
the
first-gen-ai
branch
2 times, most recently
from
81ed4a4
to
b642c4c
Compare
Co-authored-by: Liudmila Molkova <limolkova@microsoft.com>
|
@nirga are you able to fix the conflicts? I think this looks good to merge :) |
|
Thanks @joaopgrassi! I've resolved all the comments and fixed the conflicts. I think we're good to go :) |
There was a problem hiding this comment.
Changes lgtm.
A call out I have is that while we specifically only mention LLMs - ML models have been around for a long time - so isn't model observability including LLMs what we want to address with these semantic conventions? Lets discuss in our weekly SIG meeting.
|
FWIW my take is that scoping to LLMs for now is still the right call, but we should absolutely not design for only them (and I believe that's not happeing). Even in this space, there's multi-modality, language models (like embedding models) that aren't about generating text, diffusion models, etc. The good news is that a lot of the fundamentals around capturing data for various purposes are likely to be similar across many different models types. |
Work towards #327.
Changes
Replacing PRs #639 and #483. Introduces semantic conventions for modern AI systems, following discussions in OpenTelemetry LLM semconv working group.
Merge requirement checklist
[chore]